Papers with fine-grained diagnosis
Understanding Generalization in Role-Playing Models via Information Theory (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to characterize RPM generalization are inadequate . existing methods do not provide a fine-grained diagnosis of distribution shifts . |
| Approach: | They propose a reasoning-based effective mutual information difference (R-EMID) to measure RPM performance degradation in an interpretable way. |
| Outcome: | The proposed model predicts the worst-case generalization performance of RPMs and reveals how shifts contribute to degradation. |
Beyond Ranking: Fine-Grained Diagnostics and Self-Improvement for MLLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Current paradigms rely on holistic scoring and static leaderboards to disentangle fine-grained competencies. |
| Approach: | They propose a framework to shift the focus from ranking to fine-grained diagnosis. |
| Outcome: | The proposed framework surpasses the strongest baseline by 7.92%. |
Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing models fail to recall and accurately apply designated persona knowledge without explicit cues . memory-driven role-playing paradigms are attracting significant interest . |
| Approach: | They propose a memory-driven role-playing paradigm that frames persona knowledge as the LLM's internal memory store and a prompting architecture that guides structured memory retrieval and response generation. |
| Outcome: | The proposed paradigm provides a comprehensive diagnostic for four-stage role-playing abilities across 12 LLMs. |